Confidence guard: per-projection standard error, not historical RMSE - #19
Merged
Conversation
…storical RMSE The amp controller distrusted a 'no trip' verdict when the projected plateau sat within k x fit_rmse_c of the trip point — but fit_rmse_c is a model-adequacy constant across past sessions, while projection uncertainty varies enormously over a trajectory window's life. The regression already had the real thing in closed form: _project_t_inf now returns the OLS intercept standard error alongside the plateau, predict() publishes it as steady_state_se_c (floored at the sensor's 0.1 C step so a noiseless window can't claim impossible confidence), and the guard divides the margin by it, falling back to fit_rmse_c when absent. Measured on a real 43 A stretch: raw SE 0.81 C at 24 s into the window, 0.11 C by three minutes, ~0.04 C near the plateau — so the guard now distrusts exactly the early ticks the old constant scored as confident, and trusts mature projections it used to step down on. Closes #4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Issue #4: the amp controller's confidence guard treated a
will_trip: falseverdict as untrustworthy when the projected plateau was withink × fit_rmse_cof the trip point. Right idea, wrong denominator —fit_rmse_cdescribes how well the exponential model fit past sessions' ramps, a single constant, while the uncertainty of this projection, right now varies enormously over a trajectory window's life. One number for both cases made the guard too loose early (when it matters most) and too tight late (costing charge rate for nothing). The owner's charger demonstrated the late case four times on Aug 21–22: caps at 0.3–0.6 °C margins the old guard scored at 1.0–1.8 σ.Code touched
wallmonitor/thermal.py_project_t_inf()returns(t_inf, se).T_infis the intercept of the existing OLS fit onx = exp(−t/τ), so its standard error is the closed-form intercept SE:s·sqrt(1/n + mean_x²/Sxx)withs² = SSE/(n−2)— eight lines inside a function that already had every input (as the issue predicted).seisNonefor degenerate windows (n ≤ 2, or the flat-decay fallback); a noiseless flat window regresses to exactly 0.0, which is numerically true and physically overconfident, so —predict()publishessteady_state_se_con trajectory-basis forecasts, floored at 0.1 °C (the handle sensor's quantization step). Model/hypothetical bases don't get one: their uncertainty is dominated by the ambient estimate, not a regression, and pretending otherwise would be a different lie. The value rides intoforecast_samplesvia the raw payload; no schema change._project_t_infcaller (_recent_steady_ambient) ignores the SE explicitly.contrib/derate_amp_control.pysteady_state_se_cwhen the payload carries one, falling back tofit_rmse_c(older server, missing SE) — so a daemon updated ahead of its server keeps exactly the old behavior. Reason strings name which was used (… σ vs proj se/… σ vs fit rmse), so future field reports are unambiguous.--forecast-confidence-kdefault stays 2.0: against a real standard error, 2σ is a meaningful ~95 % statement rather than an arbitrary multiplier. Not silently retuned.Docs: the guard paragraph in
amp-control.mdrewritten around the per-projection error with the measured numbers; one sentence inthermal-model.md's live-forecast section.Risk
will_trip: trueand every other guard (confirm ticks, restore backoff, floors) is untouched, and alert 40 itself remains the hardware's own backstop.Verification
_project_t_infSE is >3× wider on the first quarter of a synthetic ramp than on a window that has seen the bend, exact-flat behavior documented; guard prefers the SE (wide SE + small nominal RMSE → caps, where the old guard slept), trusts a tight projection near the trip (steps up where the old guard stepped down), and falls back tofit_rmse_cbyte-for-byte when the SE is absent.contrib/backtest_derate_amp_control.py) runspredict()+decide()unchanged and now exercises the new path for free; worth a run against the production DB after a few real sessions to observe cap frequency before/after.Deploy: server (
git pull+ restart for the poller's forecasts) and the amp-control daemon pick this up independently; either order is safe thanks to the fallback.Closes #4.
🤖 Generated with Claude Code